Image coding device and image decoding device

ABSTRACT

To provide an image coding device having high coding efficiency and an image decoding device. A plurality of prediction procedures that uses various types of correlations between pixels are adaptively applied with a coded signal as a reference. With respect to an input pixel, first residual and prediction information are obtained by a first prediction unit that carries out in-screen prediction and the like, second residual and prediction information are obtained by a second prediction unit for predicting a first residual as a serial additional process, and third residual and prediction information are obtained by a third prediction unit for directly predicting a pixel to be coded as a parallel additional process on the input pixel. Which of either the second residual or the third residual to code is determined based on the coding cost, and the signal is switched in the first switching unit.

The present application claims priority of Japanese patent applicationSerial No. 2011-001136, filed Jan. 6, 2011, the content of which ishereby incorporated by reference in its entirety.

TECHNICAL FIELD

The present invention relates to an image coding device for predicting acoding target by adaptively applying a plurality of different predictionprocedures on correlative information and coding a prediction residual,and an image decoding device.

BACKGROUND ART

A method of enhancing the coding efficiency in the conventional imagecoding includes a method of reducing spatial redundancy. The predictionof a target block is carried out using an adjacent coded pixel in anintra prediction coding in H.264. H.264 is disclosed in Kadono et al.,“H.264/AVC text Impress standard text series”, Impress network businesscompany, 2004, and the like.

Japanese Patent Application Laid-Open No. 2007-043651 proposes searchingfor a similar area from a coded region with an adjacent coded pixel as atemplate, and using an adjacent region at the same positionalrelationship as the target block as a prediction value of the targetblock.

Japanese Patent Application Laid-Open No. 2007-074725 proposes dividingthe target block into plurals, coding and decoding one part of thedivided block, and using the pixel in which one part of the block isdecoded for the prediction on the remaining pixels.

Furthermore, Japanese Patent Application Laid-Open No. 2009-049969proposes using not only the adjacent pixels between blocks but also theadjacent pixel in the target block as a prediction value to use highcorrelation between adjacent pixels.

CITATION LIST Patent Literature

-   Patent Literature 1 “Japanese Patent Application Laid-Open No.    2007-043651”-   Patent Literature 2 “Japanese Patent Application Laid-Open No.    2007-074725”-   Patent Literature 3 “Japanese Patent Application Laid-Open No.    2009-049969”

Non Patent Literature

-   Non Patent Literature 1 Kadono et al., “H.264/AVC text Impress    standard text series”, Impress network business company, (P. 108    FIG. 5-3, pp. 203-205) 2004

SUMMARY OF INVENTION Technical Problem

The Intra prediction of H.264 generates the prediction value of thetarget block with the coded neighboring pixel as a reference, and hencethe prediction error becomes larger the more distant from the pixelserving as the reference of prediction.

In the technique disclosed in Japanese Patent Application Laid-Open No.2007-043651, information representing the location for instructing theblock to use in the prediction of the target block does not need to bestored, but the prediction accuracy may not be sufficient since theblocks may not be similar even if the adjacent pixels to become atemplate are similar.

In the technique disclosed in Japanese Patent Application Laid-Open No.2007-074725, one part of the block to be coded first has the sameproblem as H.264 when predicting the divided target block.

In the technique disclosed in Japanese Patent Application Laid-Open No.2009-049969, if the adjacent pixel is not coded, a quantization errormay be propagated since the differential value with the adjacent pixelin the original image is coded.

In view of solving the problems of the related art, it is an object ofthe present invention to provide an image coding device having highcoding efficiency.

Solution to Problem

In order to accomplish the object, the feature of this invention is thatan image coding device for coding an input image for every unit block incoding using prediction, the image coding device comprising: a firstprediction unit for determining first prediction information forpredicting a block to be coded using a reconstructed pixel signalreconstructed using a coded signal; a first compensation unit forobtaining a first prediction signal of said block to be coded from saidfirst prediction information and said reconstructed pixel signal; afirst differential unit for obtaining a difference between a signal ofsaid block to be coded and said first prediction signal as a firstresidual signal; a second prediction unit for determining secondprediction information as a prediction coefficient for predicting saidfirst residual signal using a reconstructed residual signalreconstructed using the coded signal; a second compensation unit forobtaining a second prediction signal of said first residual signal fromsaid second prediction information and said reconstructed residualsignal; a second differential unit for obtaining a difference betweensaid first residual signal and said second prediction signal as a secondresidual signal; a third prediction unit for determining thirdprediction information as a prediction coefficient for predicting saidblock to be coded using said reconstructed pixel signal; a thirdcompensation unit for obtaining a third prediction signal of said blockto be coded from said third prediction information and saidreconstructed pixel signal; a third differential unit for obtaining adifference between a signal of said block to be coded and said thirdprediction signal as a third residual signal; and a switching unit forswitching and selecting one of either said second residual signal orsaid third residual signal for every unit block, wherein orthogonaltransformation, quantization, and coding are performed on the selectedresidual, and said reconstructed residual signal and said reconstructedpixel signal are obtained using prediction information corresponding tothe selected residual to code the prediction information.

Advantageous Effects of Invention

According to the image coding device of the present invention, highcoding efficiency is obtained by adaptively applying a plurality ofprediction procedures that use various types of correlations betweenpixels with a coded signal as a reference.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a function block diagram of an image coding device of thepresent invention;

FIG. 2 is a function block diagram of an image decoding devicecorresponding to the image coding device of FIG. 1;

FIG. 3 is a view describing an example of a relationship between aprediction reference signal used when applying a second prediction unit,and the like and a predicted signal;

FIG. 4 is a view describing an example of a relationship of arepresentative value used when applying the second prediction unit andthe like and a predicted signal, and a small region for calculating therepresentative value; and

FIG. 5 is a view describing an example of a relationship of arepresentative value used when applying a third prediction unit and thelike, a predicted signal, and a coded neighboring region and the like.

DESCRIPTION OF EMBODIMENTS

Hereinafter, the present invention will be described in detail withreference to the drawings. FIG. 1 shows a function block diagram of animage coding device according to the present invention. The image codingdevice of the present invention has a configuration in which thefunction of predicting the residual signal in the unit block from thecoded residual signal in parallel and in series is added with respect toan image coding device for performing orthogonal transformation,quantization, and coding on a residual signal obtained by performing adifferential process with each pixel predicted from a coded pixel withrespect to each pixel of a unit block configured by a plurality ofpixels to perform coding for every unit block, so that it can beappropriately selected.

When viewed from the input image signal (non-coded pixel signal) side,the parallel addition is the relationship of function block groups 100(first prediction unit 10, compensation unit 11, accumulation unit 12,differential unit 13, and addition unit 14) and 300 (third predictionunit 30, compensation unit 31, accumulation unit 32, differential unit33, and addition unit 34) shown in FIG. 1; and the serial addition isthe relationship of the function block groups 100 and 200 (secondprediction unit 20, compensation unit 21, accumulation unit 22,differential unit 23, and addition unit 24). The present invention hascharacteristics in the coordination of such function block groups.

In other words, as shown in FIG. 1, as a configuration responsible forthe main portion of its function, the image coding device of the presentinvention is configured to include, with respect to an image codingdevice including a transformation unit 1 for transforming a residualsignal to a frequency region in orthogonal transformation and obtainingan orthogonal transformation coefficient, a quantization unit 2 forquantizing the orthogonal transformation coefficient to obtain aquantization value, a coding unit 5 for variable length coding thequantization value and the applied prediction information and obtainingcoded information, an inverse quantization unit 3 for inverse quantizingthe quantization value and obtaining an orthogonal transformationcoefficient, an inverse transformation unit 4 for inverse transformingthe inverse quantized orthogonal transformation coefficient andobtaining the residual signal, a first prediction unit 10 fordetermining the prediction information for reducing redundancy from thepixel signal, and a first compensation unit 11 for re-constructing theprediction signal from the prediction information, second and thirdprediction units 20, 30 for predicting the signal from the pixel signalor the residual signal, second and third compensation units 21, 31 forobtaining the prediction signal, and first, second, and third switchingunits 40, 41, 42 for appropriately switching both signals andcorresponding/accompanying information.

As shown in FIG. 2, as a configuration responsible for the main portionof its function, the image decoding device of the present invention isconfigured to include, with respect to an image decoding deviceincluding a decoding unit 6 for variable length decoding the codedinformation and obtaining the quantization value, an inversequantization unit 7 for inverse quantizing the quantization value andobtaining the orthogonal transformation coefficient, an inversetransformation unit 8 for inverse transforming the inverse quantizedorthogonal transformation coefficient and obtaining the residual signal,and a first compensation unit 51 for re-constructing the predictionsignal from the stored first prediction information and the decodedpixel signal, a third prediction unit 70 for predicting the signal fromthe pixel signal and second and third compensation units 61, 71 forobtaining the prediction signal, and first, second, and third switchingunits 80, 81, 82 for appropriately switching both signals andcorresponding/accompanying information.

Each function block of the image coding device of the present inventionshown in FIG. 1 will be described below.

The first differential unit 13 calculates the difference between aninput image signal (pixel signal) and a first prediction signalpredicted from the coded pixel sent from the first compensation unit 11.The first residual signal obtained by taking the difference is sent tothe second prediction unit 20 and the second differential unit 23.

The second differential unit 23 calculates the difference between thefirst residual signal sent from the first differential unit 13 and theprediction signal predicted from the coded residual signal sent from thesecond compensation unit 21. The second residual signal obtained bytaking the difference is sent to the transformation unit 1 if selectedby the first switching unit 40 as will be described later.

The third differential unit 33 calculates the difference between theinput image signal (pixel signal) and the third prediction signalpredicted from the coded pixel sent from the third compensation unit 31.The third residual signal obtained by taking the difference is sent tothe transformation unit 1 if selected by the first switching unit 40 aswill be described later.

The first addition unit 14 re-constructs the coded pixel by calculatingthe sum of the residual signal sent from the second addition unit 24 andthe prediction signal sent from the first compensation unit 11. Thecoded pixel signal (reconstructed pixel signal) obtained by adding issaved in the first accumulation unit 12 to be referenced and used by thefirst prediction unit 10 and the first compensation unit 11.

The second addition unit 24 calculates the sum of the reconstructedresidual signal sent from the inverse transformation unit 4 through thesecond switching unit 41 and the prediction signal sent from the secondcompensation unit 21 to reconstruct a first residual signal. The firstresidual signal obtained by adding is sent to the second accumulationunit 22 and the first addition unit 14. In the second accumulation unit22, the first residual signal is saved so as to be referenced and usedby the second prediction unit 20 and the second compensation unit 21.

The third addition unit 34 calculates the sum of the reconstructedresidual signal sent from the inverse transformation unit 4 through thesecond switching unit 41 and the third prediction signal sent from thethird compensation unit 31 to reconstruct a coded pixel. The coded pixelsignal (reconstructed pixel signal) obtained by adding is sent to andsaved in the third accumulation unit 32. The saved coded pixel signal isreferenced and used by the third prediction unit 30 and the thirdcompensation unit 31.

Either one of the second or third residual signal sent from the seconddifferential unit 23 or the third differential unit 33 is selected bythe first switching unit 40 and input to the transformation unit 1. Theselected residual signal is transformed to the frequency region byorthogonal transformation, and the transformation coefficient obtainedby orthogonal transformation is output to the quantization unit 2. Theorthogonal transformation may be DCT or approximate transformation ofDCT, DWT, or the like.

In the image coding device of the present invention, each picture(frame) of the input image is divided into unit blocks configured bypixels of predefined number (e.g., 32×32 pixels, 16×16 pixels, 8×8pixels, 4×4 pixels, or combination thereof), so that coding is carriedout for every unit block.

The quantization unit 2 quantizes the transformation coefficient sentfrom the transformation unit 1. The quantization value obtained byquantization is output to the coding unit 5 and the inverse quantizationunit 3. The quantization parameter used in the quantization process canbe set as a combination of constant values. The bit rate to output maybe maintained constant by performing the control according to the amountof information of the transformation coefficient.

Alternatively, as one embodiment of the quantization parameter in thequantization unit 2, the overall performance can be enhanced byperforming a control such that the quantization error becomes small withrespect to the signal (channel) to become the reference of prediction.The signal to become the reference of prediction will be describedlater.

The coding unit 5 codes the quantized transformation coefficient sentfrom the quantization unit 2, and the first to third predictioninformation sent from the first to third prediction units, and outputsas the coded information. The coding may use variable length code orarithmetic code that removes redundancies between codes.

The inverse quantization unit 3 performs the procedure opposite of thequantization process in the quantization unit 2 to inversely quantizethe quantized transformation coefficient sent from the quantization unit2. The transformation coefficient including the quantization errorobtained through the inverse quantization is sent to the inversetransformation unit 4.

The inverse transformation unit 4 performs the procedure opposite of theorthogonal transformation in the transformation unit 1 to inverselyorthogonal transform the transformation coefficient including thequantization error sent from the inverse quantization unit. The residualsignal including the quantization error obtained by inversetransformation is sent to either the second addition unit 24 or thethird addition unit 34 through the second switching unit 4.

The second switching unit 41 carries out the selection process ofsending the residual signal sent from the inverse transformation unit 4to the second addition unit 24 if originating from the second residualsignal subjected through the processes of the function block groups 100and 200, and sending the residual signal to the third addition unit 34if originating from the third residual signal subjected to the processof the function block group 300.

Such selection process is carried out in cooperation with the firstswitching unit 40 and the third switching unit 42, and is carried out bythe control unit or the like for controlling the entire function blockof the coding device not shown in FIG. 1. As will be described below,according to such cooperation, the first and second predictioninformation are passed by the third switching unit 42 if the secondresidual signal is passed by the first and second switching units 40,41, so that the second residual signal and the first and secondprediction information are coded as a set. The third predictioninformation is passed by the third switching unit 42 if the thirdresidual signal is passed by the first and second switching units 40,41, so that the third residual signal and the third predictioninformation are coded as a set. Therefore, the first, second, and thirdswitching units 40, 41, 42 carry out switching in cooperation so thatthe corresponding signal and information are coded together. This issimilar in the first, second, and third switching units 80, 81, and 82on the decoder side.

The first prediction unit 10 determines the prediction information forreducing the redundancy of the input pixel, and determines the firstprediction information for predicting and approximating the input signalbased on the coded pixel (reconstructed pixel) including thequantization error saved in the first accumulation unit 12. Thedetermined first prediction information is sent to the firstcompensation unit 11, and also sent to the coding unit 5 through thethird switching unit 42. Various methods conventionally used can beapplied for the first prediction.

By way of example, when using the intra prediction of the internationalstandard H.264, the intra prediction mode of individually coding in eachintra prediction mode, and minimizing the coding cost calculated as aweighted sum or the like from the code amount and the distortion amountis selected as the first prediction information. The details on themethod of minimizing the coding cost are described in Kadono et al.,“H.264/AVC text Impress standard text series”, Impress network businesscompany, 2004.

The first compensation unit 11 predicts redundancy and reconstructs thefirst prediction signal, where the prediction signal of the relevantregion is generated from the first prediction information sent from thefirst prediction unit 10 and the coded pixel signal saved in the firstaccumulation unit 12. The first prediction signal is sent to the firstdifferential unit 13 and the first addition unit 14 in the encoder, andonly to the first addition unit 54 in the decoder.

The calculation of the residual signal (second and third residualsignals) input to the transformation unit 1 through the function blockgroups 200 and 300, which are characteristic configurations of the imagecoding device of the present invention, will be described with theassociated function blocks in FIG. 1.

In calculating the residual signal, the calculation of the secondprediction signal and the third prediction signal becomes necessary. Thecalculation of the second prediction signal is carried out by theconfiguration of arranging the second prediction unit 20 and the secondcompensation unit 21 in the function block group 200 in correspondencewith the first prediction unit 10 and the first compensation unit 11 inthe function block group 100. The calculation of the third predictionsignal is carried out by the configuration of arranging the thirdprediction unit 30 and the third compensation unit 31 in the functionblock group 300.

The second residual signal and the third residual signal are comparedwith regards to the coding cost corresponding thereto, and the one withsmaller coding cost is exclusively selected by the first switching unit40. The coding cost corresponding to the second residual signal is thecoding cost of the second residual signal, the first predictioninformation, and the second prediction information, and the coding costcorresponding to the third residual signal is the coding cost of thethird residual signal and the third prediction information.

In the present invention, an embodiment of evaluating also a case inwhich the residual signal is forcibly eliminated in the calculation ofthe coding cost (case in which value of residual signal is zero) usingthe fact that the prediction residual tends to become smaller since theprediction accuracy is higher due to the improvement of the predictionunit is desirable. In other words, in the embodiment, transformation,quantization, and coding are performed by comparing a total of fourtypes of values, the coding cost (two types) corresponding to the secondand third residual signals and the coding cost (two types) correspondingto the residual signals of when the second and third residual signalsare zero, and adopting that of minimum cost. The relevant process can becarried out as an additional process in the first switching unit 40, andthe like.

In the embodiment, the coding in zero residual that is advantageous forthe total coding cost can be adopted when the code amount can be greatlyreduced even if the amount of distortion is slightly increased with theresidual as zero than when the already sufficiently small predictionresidual is coded and the image quality is slightly enhanced whileslightly suppressing the amount of distortion.

Each function of the second prediction unit 20, the second compensationunit 21, the third prediction unit 30, and the third compensation unit31 will be hereinafter described.

The second prediction unit 20 determines the second predictioninformation for reducing the redundancy remaining in the first residualsignal in the unit block. The second prediction unit 20 calculates theprediction coefficient for approximating and predicting the firstresidual signal sent from the first differential unit 13 with themethod, to be described later, from the first residual signal includingthe quantization error saved in the second accumulation unit 22. Thecalculated prediction coefficient is sent to the second compensationunit 21 and to the coding unit 5 through the third switching unit 42 asthe second prediction information.

However, the second prediction unit 20 and the second compensation unit21 are omitted if the input first residual signal is flat. Whether flator not is determined from whether or not the variance (dispersion valueetc.) of each value of the unit block of the first residual signalsatisfies a predetermined standard such as smaller, or smaller than orequal to a predetermined value. If determined as flat, the signal havingthe value of the first residual signal is used for the second residualsignal to be sent to the first switching unit 40 according to the aboveomission. The processes of determination and omission are carried out bythe control unit (not shown). In this case, the flag informationindicating that the omission is carried out is used for the secondprediction information, so that control can be performed to omit theprocesses of second prediction and compensation on the decoding side.

The calculation of the prediction coefficient by the second predictionunit 20, and the generation order of the second prediction signal by thesecond compensation unit 21 for cases other than the above, that is,when the input first residual signal is not flat will be describedbelow.

First, it is to be noted that the first residual signal saved in thesecond accumulation unit 22 used to generate the second predictionsignal is the first residual signal in the signal of a block at the sameposition in the frame of the same time as the block to be coded and thealready coded different channel, as conceptually shown in FIG. 3. Eachsignal channel may use each color space axis in the signal mapped to anarbitrary color space such as RGB signal, YUV signal, or YCbCr signal.FIG. 3 shows an example of the RGB signal, where the already coded block(a2) in the G signal frame shown in (a1) is such that the first residualsignal of the G signal in the block is reconstructed and saved in thesecond accumulation unit 22 to be used as a reference. The block to becoded (b2) is the block at the same position in the frame (b1) of the Rsignal in the same time frame as the relevant (a2). When coding the Rsignal block (b2), as for the reconstructed first residual signal to bereferenced in the second accumulation unit 22, the first residual signalin the coded G signal block (a2) is adopted.

The signal (channel) to become the reference of prediction that controlsthe quantization parameter such that the quantization error becomessmall in the quantization unit 2 is a signal (channel) coded before thechannel to be coded of the block to be coded such as the G signal shownin the example of FIG. 3. In the present invention, the predicted signalof a different channel is coded using the residual signal in theprediction reference signal, and hence the predicted signal can also bepredicted at high accuracy by having the prediction reference signal athigh image quality such that the quantization error becomes small, whichleads to reduction in the total coding cost. The signal such as the Gsignal in the example of FIG. 3 is generally referred to as theprediction reference signal, and the signal such as the R signal isreferred to as the predicted signal.

In the second prediction unit 20, when the second prediction unitperforms correction with affine prediction as an example of correctionby the prediction coefficient, a representative value g_(i) with respectto each pixel position is calculated for every small region inside theblock, as hereinafter described, from the coded first residual signalwith the affine coefficient a_(j) (1≦j≦2) as the second predictioncoefficient, and the residual signal r_(i) of the relevant block ispredicted with (Math. 1).r _(i) =a ₁ g _(i) +a ₂1≦i≦n  [Math. 1]

Here, n represents the number of pixels in the block.

The representative value g_(i) may use the value of the first residualsignal at the relevant pixel position as is, but preferably uses arepresentative value serving as a value calculated from the value of thefirst residual signal in each small region from the standpoint of noiseresistance, as will be described later.

An example of the representative value g_(i) and the residual signalr_(i) to be predicated is shown in FIG. 4. In FIG. 4, (a2) and (b2) arethe same as FIG. 3, and show the block of the coded G signal and theblock of the R signal to be coded, respectively. In FIG. 4, an examplein which the coded first residual signal in (a2) is a residual signal bythe prediction mode 0 for predicting in the vertical direction of theprediction modes 0 to 8 of the 4×4 intra prediction of the H.264 in thefunction block group 100. In other words, A1 (g₁ to g₄), A2 (g₅ to g₈),A3 (g₉ to g₁₂) and A4 (g₁₃ to g₁₆) show the representative value g_(i)of the first residual signal at the four pixel positions in the verticaldirection predicted from the same adjacent pixel of the upper end of(a2). In this example, A1 to A4 are set as small regions. The residualsignals B1 (r₁ to r₄), B2 (r₅ to r₃), B3 (r₉ to r₁₂) and B4 (r₁₃ to r₁₆)at the same position are predicted using the representative valuedefined on each pixel position.

The calculation of the representative value desirably providesresistance to noise by using Gaussian filter or the like when thecorrelation of the input signal is independent (e.g., independent amongR, G, B signals for RGB signal) such as a three plate type camera. Also,a filter adapted to the noise characteristics corresponding to the firstprediction information may be applied. In other words, in the example ofFIG. 4, since prediction is made in the vertical direction, if the noiseis superimposed on the first residual signal of the pixel at theposition of g₆, it is not preferable if the influence of the noise isstretched in the horizontal direction (e.g., position of g₂, or positionof gio) by the filter processing with respect to the first residualsignal calculated by the prediction mode 0 in the vertical direction.Thus, in the example of (a2), the filter processing is to beindividually carried out for every pixel in each small region A1 to A4of each vertical direction. In the small region A1, the filterprocessing is performed on the residual signals at the positions of g₁to g₄.

The filter application adapted to the noise characteristicscorresponding to the first prediction information is described in theexample of the prediction mode 0 of the 4×4 intra prediction of H.264,but it is apparent that the filter processing can be similarly appliedfor every region corresponding to the direction of the predictioninformation even in other examples. For instance, in another predictionmode of the 4×4 intra prediction of H.264, the filter is applied on eachregion in the horizontal direction in the case of the prediction mode 1and the entire 4×4 unit block of regional distinction, which is assumedas one region, in the case of prediction mode 2. The filter is appliedon each region corresponding to the diagonal prediction direction forthe prediction modes 3 to 8.

The prediction coefficient a_(j) is estimated such that the weightedsquare sum of the prediction error becomes a minimum. Specifically, oneexample of a calculation method for the prediction coefficient a_(j)will be described. The weighted square sum E of the prediction error isexpressed with (Math. 2).

$\begin{matrix}{E = {\sum\limits_{i}^{\;}{w_{i}( {{a_{1}g_{i}} + a_{2} - r_{i}} )}^{2}}} & \lbrack {{Math}.\mspace{14mu} 2} \rbrack\end{matrix}$

In this case, the partial differentiation of the square error E by thecoefficient a_(j) is expressed with (Math. 3).

$\begin{matrix}{{{\frac{1}{2}\frac{\partial E}{\partial a_{1}}} = {\sum\limits_{i}^{\;}{w_{i}{g_{i}( {{a_{1}g_{i}} + a_{2} - r_{i}} )}}}}{{\frac{1}{2}\frac{\partial E}{\partial a_{2}}} = {\sum\limits_{i}^{\;}{w_{i}( {{a_{1}g_{i}} + a_{2} - r_{i}} )}}}} & \lbrack {{Math}.\mspace{14mu} 3} \rbrack\end{matrix}$

In order to minimize the square error E, (Math. 3) needs to be 0, andthus the coefficient a_(j) is obtained with (Math. 4). However, thenotation of the suffix i is omitted on the grounds of space.

$\begin{matrix}{\begin{pmatrix}a_{1} \\a_{2}\end{pmatrix} = \begin{pmatrix}{- \frac{{\sum{{wg}{\sum{wr}}}} - {\sum{w{\sum{wgr}}}}}{{\sum{{wg}^{2}{\sum w}}} - ( {\sum{wg}} )^{2}}} \\\frac{{\sum{{wg}^{2}{\sum{wr}}}} - {\sum{{wg}{\sum{wgr}}}}}{{\sum{{wg}^{2}{\sum w}}} - ( {\sum{wg}} )^{2}}\end{pmatrix}} & \lbrack {{Math}.\mspace{14mu} 4} \rbrack\end{matrix}$

Alternatively, when the second prediction unit performs correction inproportion, similarly as in the case of the affine prediction, therepresentative value g_(i) is calculated for every small region in theblock from the coded first residual signal with the proportionalitycoefficient a_(j) (j=1) as the second prediction coefficient, and theresidual signal r_(i) of the relevant block is predicted with (Math. 5).In the calculation of the representative value, it is desirable toprovide resistance to noise by using the Gaussian filter, or the likesimilar to the description made above.r _(i) =a ₁ g _(i)1≦i≦n  [Math. 5]

Here, n is the number of pixels in the block. The prediction coefficienta_(j) is estimated to minimize the weighted square sum of the predictionerror. One example of a calculation method will be specificallydescribed for the prediction coefficient a_(j). The weighted square sumE of the prediction error is expressed with (Math. 6).

$\begin{matrix}{E = {\sum\limits_{i}^{\;}{w_{i}( {{a_{1}g_{i}} - r_{i}} )}^{2}}} & \lbrack {{Math}.\mspace{14mu} 6} \rbrack\end{matrix}$

In this case, the partial differentiation of the square error E by thecoefficient a_(j) is expressed with (Math. 7).

$\begin{matrix}{{\frac{1}{2}\frac{\partial E}{\partial a_{1}}} = {\sum\limits_{i}^{\;}{w_{i}{g_{i}( {{a_{1}g_{i}} - r_{i}} )}}}} & \lbrack {{Math}.\mspace{14mu} 7} \rbrack\end{matrix}$

In order to minimize the square error E, (Math. 7) needs to be 0, andthus the multiplier a_(j) is obtained with (Math. 8). However, thenotation of the suffix i is omitted on the grounds of space.

$\begin{matrix}{a_{1} = \frac{\sum{wgr}}{\sum{wg}^{2}}} & \lbrack {{Math}.\mspace{14mu} 8} \rbrack\end{matrix}$

The second prediction unit 20 may have the polynomial equation of two ormore orders as a prediction expression. Alternatively, the secondprediction unit 20 may have a plurality of prediction expressions, andmay appropriately switch the prediction expression. When the respectiveprediction expression is applied, a combination that minimizes thecoding cost is selected. In this case, the information representing theselected prediction expression and the prediction coefficient are sentto the coding unit 5 as the second prediction information.

In either case, the weight coefficient w_(i) can be arbitrarily set. Inthe first prediction unit and the first compensation unit, it isdesirable to maintain the image quality high at the lower end and theright end of the block (in the example of (b2) of FIG. 4, lower end ispositions of r₄, r₉, r₁₂, and r₁₆, and right end is positions of r₁₃,r₁₄, r₁₅, and r₁₆) since these ends are used for the prediction and thecompensation of the neighboring block coded at the time point later thanthe block to be coded. In other words, the weight coefficients of thelower end and the right end of the block are made relatively high tomaintain the image quality of the relevant location high.

Only one set of prediction coefficients may be obtained for the entireunit block, or the prediction coefficient may be obtained for each smallregion performed with filter processing. In the example of FIG. 4, a setof prediction coefficients for predicting the entire block (b2) from theentire block (a2) may be obtained, or four sets of predictioncoefficients for predicting the corresponding regions B1, B2, B3 and B4from the small regions A1, A2, A3, and A4 may be obtained. From thestandpoint of code amount, it is preferable to set to obtain only oneset of prediction coefficients for the entire unit block.

The second compensation unit 21 reconstructs the second predictionsignal, and obtains the second prediction signal from the secondprediction information from the second prediction unit 20 and the firstresidual signal from the second addition unit 24. The generated secondprediction signal is output to the second addition unit 24 and thesecond difference unit 23.

In the above-described example where the second prediction unit 20 usesthe affine prediction for the prediction function and the secondprediction information is configured from the prediction coefficienta_(j), the prediction signal is generated with (Math. 9).r _(i) =a ₁ g _(i) +a ₂  [Math. 9]

The second prediction signal is sent to the second differential unit 23and the second addition unit 24 in the encoder, and only to the secondaddition unit 64 in the decoder.

The third prediction unit 30 determines the third prediction informationfor reducing the redundancy that exists in the input pixel in the unitblock.

The third prediction unit 30 first calculates the prediction coefficientfor approximating the pixel signal with the method, to be describedlater, from the coded pixel signal (reconstructed pixel signal) saved inthe third accumulation unit 32. The calculated prediction coefficient issent to the third compensation unit 31 and the coding unit 5 as thethird prediction information.

The calculation of the prediction coefficient by the third predictionunit 30, and the generation procedure of the third prediction signal bythe third compensation unit 31 will be described below. It is to benoted that in the third prediction unit 30 as well, there is a codedblock (a2) in a different channel in the block of the same position inthe same time frame in advance when coding the block (b2) to be coded,as described using FIG. 3 in the second prediction unit 20.

A case in which the third prediction unit 30 predicts the pixel signalfrom the coded neighboring pixel will be described using FIG. 5 as oneexample in which the third prediction unit 30 performs correction by theprediction coefficient. In FIG. 5, the G signal frame (a1) including thecoded block (a2) and the R signal frame (b1) including the block (b2) tobe coded in which the time and position are common with the (a2) but thesignal channel is different are shown, similarly as used in the exampleof FIG. 3.

In the third prediction unit 30, the already coded neighboring pixel(b3) being in the vicinity adjacent to the block (b2) to be coded andbeing already coded, and the coded neighboring pixel (a3) of the sameposition as the (b3) in the frame (a1) are used. Assuming that thecorrespondence relationship of the coded neighboring pixels (a3) and(b3) corresponds to the correspondence relationship of the relevantblocks (a2) and (b2), the prediction coefficient similar to the secondprediction unit 20 is calculated from the coded neighboring pixels (a3)and (b3), and applied to the blocks (a2) and (b2).

In other words, as shown in (c1) of FIG. 5( c), the representative valueg_(i) in (Math. 1) to (Math. 8) is calculated based on each pixel signalvalue of the region (a3). The signal r_(i) of (b3) at the same positionis predicted using the representative value g_(i) calculated withrespect to each pixel position of the (a3) to obtain the predictioncoefficient. Then, as shown in (c2), the block (b2) to be coded ispredicted from the coded block (a2) using the prediction coefficient forpredicting (b3) from the coded neighboring pixel region (a3).

The difference with the second prediction unit 20 is whether theprocessing target region is the position of the block to be coded or thecoded pixel region in the vicinity thereof, and whether the processingtarget is the first residual signal or the pixel signal, and whether ornot to hold the prediction coefficient. In other words, the thirdprediction unit 30 directly predicts the pixel signal as shown in (c2),and also does not need to hold the prediction coefficient by carryingout the process similarly in the decoding process since the predictioncoefficient can be calculated from (a3) and (b3) or the coded regions asshown in (c1), and hence the code amount of the prediction informationcan be reduced.

The third prediction unit 30 can apply different prediction coefficientsfor every pixel signal without increasing the code amount of theprediction coefficient as it does not hold the prediction coefficient.For instance, when calculating the prediction coefficient for each pixelsignal of the processing target block, each small region in the targetblock or each target block, the prediction coefficient can be changed bysetting the weight coefficient according to the distance from the targetpixel to the neighboring pixel and/or the difference between the targetpixel value and the neighboring pixel. The correspondence can beappropriately set by partitioning either one of or both of the regionsof the neighboring pixel or the block to be coded into small regions,and the prediction coefficient may be set for every correspondence. Thesetting may be to obtain the prediction coefficient with respect to eachsmall region of the block to be coded. But the increase in code amountdoes not occur even when obtaining a plurality of sets of predictioncoefficients in the entire block, as opposed to the case of the secondprediction unit 20.

Furthermore, the third prediction unit 30 can maintain the predictionaccuracy high without causing quantization error in the predictioncoefficient as it does not hold the prediction coefficient. As the thirdprediction unit 30 does not need to code the prediction coefficient, theaccuracy can be enhanced using the fact that the code amount does notrelatively increase even if a polynomial equation of higher order isused. For instance, when the third prediction unit 30 performscorrection with the quadratic expression as an example of the correctionby the prediction coefficient, similarly as in the case of the secondprediction unit 20, the representative value g_(i) is calculated forevery small region or the entire region of the neighboring pixel fromthe coded pixel signal positioned in the vicinity of the block to becoded, and the coded pixel signal r_(i) positioned in the vicinity ofthe target block is predicted with (Math. 10), assuming the coefficienta_(j) (1≦j≦3) is the third prediction coefficient. In the calculation ofthe representative value, it is desirable to give resistance to noise byusing the Gaussian filter and the like in the relevant region or theentire neighboring pixel region.r _(i) =a ₁ g _(i) ² +a ₂ g _(i) +a ₃1≦i≦n  [Math. 10]

Here, n is the number of pixels in the block. The prediction coefficienta_(j) is estimated to minimize the weighted square sum of the predictionerror. One example of the calculation method for the predictioncoefficient a_(j) will be specifically described. The weighted squaresum E of the prediction error is expressed with (Math. 11).

$\begin{matrix}{E = {\sum\limits_{i}^{\;}{w_{i}( {{a_{1}g_{i}^{2}} + {a_{2}g_{i}} + a_{3} - r_{i}} )}^{2}}} & \lbrack {{Math}.\mspace{14mu} 11} \rbrack\end{matrix}$

In this case, the partial differentiation of the square error sum E bythe coefficient a_(j) is expressed with (Math. 12).

$\begin{matrix}{{\frac{1}{2}\frac{\partial E}{\partial a_{1}}} = {\sum\limits_{i}^{\;}{w_{i}{g_{i}^{2}( {{a_{1}g_{i}^{2}} + {a_{2}g_{i}} + a_{3} - r_{i}} )}}}} & \lbrack {{Math}.\mspace{14mu} 12} \rbrack \\{{\frac{1}{2}\frac{\partial E}{\partial a_{2}}} = {\sum\limits_{i}^{\;}{w_{i}{g_{i}( {{a_{1}g_{i}^{2}} + {a_{2}g_{i}} + a_{3} - r_{i}} )}}}} & \; \\{{\frac{1}{2}\frac{\partial E}{\partial a_{3}}} = {\sum\limits_{i}^{\;}{w_{i}( {{a_{1}g_{i}^{2}} + {a_{2}g_{i}} + a_{3} - r_{i}} )}}} & \;\end{matrix}$

In order to minimize the squarer error E, each equation in (Math. 12)needs to become 0, and hence the multiplier a_(j) is obtained with(Math. 13). However, the notation of the suffix i is omitted on thegrounds of space.

$\begin{matrix}{\begin{pmatrix}a_{1} \\a_{2} \\a_{3}\end{pmatrix} = \begin{pmatrix}{- \frac{\begin{matrix}{{( {{\sum{{wg}^{3}{\sum{wgr}}}} - {\sum{{wg}^{2}{\sum{{wg}^{2}r}}}}} ){\sum w}} +} \\{{( {( {\sum{wg}^{2}} )^{2} - {\sum{{wg}^{3}{\sum{wg}}}}} ){\sum{wr}}} -} \\{{\sum{{wg}^{2}{\sum{{wg}{\sum{wgr}}}}}} + {\sum{{wg}^{2}{\sum{{wg}^{2}r}}}}}\end{matrix}}{\begin{matrix}{{( {{\sum{{wg}^{4}{\sum{wg}^{2}}}} - ( {\sum{wg}^{3}} )^{2}} ){\sum w}} - {\sum{{wg}^{4}{\sum{wg}^{2}}}} +} \\{{2{\sum{{wg}^{3}{\sum{{wg}^{2}{\sum{wg}}}}}}} - ( {\sum{wg}^{2}} )^{3}}\end{matrix}}} \\\frac{\begin{matrix}{{( {{\sum{{wg}^{4}{\sum{wgr}}}} - {\sum{{wg}^{3}{\sum{{wg}^{2}r}}}}} ){\sum w}} +} \\{{( {{\sum{{wg}^{3}{\sum{wg}^{2}}}} - {\sum{{wg}^{4}{\sum{wg}}}}} ){\sum{wr}}} -} \\{{( {\sum{wg}^{2}} )^{2}{\sum{wgr}}} + {\sum{{wg}^{2}{\sum{{wg}{\sum{{wg}^{2}r}}}}}}}\end{matrix}}{{( {{\sum{{wg}^{4}{\sum{wg}^{2}}}} - ( {\sum{wg}^{3}} )^{2}} ){\sum w}} - {\sum{{wg}^{4}{\sum{wg}^{2}}}} +} \\{{2{\sum{{wg}^{3}{\sum{{wg}^{2}{\sum{wg}}}}}}} - ( {\sum{wg}^{2}} )^{3}} \\\frac{\begin{matrix}{{( {{\sum{{wg}^{4}{\sum{wg}^{2}}}} - ( {\sum{wg}^{3}} )^{2}} ){\sum{wr}}} +} \\{{( {{\sum{{wg}^{3}{\sum{wg}^{2}}}} - {\sum{{wg}^{4}{\sum{wg}}}}} ){\sum{wgr}}} +} \\{( {{\sum{{wg}^{3}{\sum{wg}}}} - ( {\sum{wg}^{2}} )^{2}} ){\sum{{wg}^{2}r}}}\end{matrix}}{\begin{matrix}{{( {{\sum{{wg}^{4}{\sum{wg}^{2}}}} - ( {\sum{wg}^{3}} )^{2}} ){\sum w}} - {\sum{{wg}^{4}{\sum{wg}^{2}}}} +} \\{{2{\sum{{wg}^{3}{\sum{{wg}^{2}{\sum{wg}}}}}}} - ( {\sum{wg}^{2}} )^{3}}\end{matrix}}\end{pmatrix}} & \lbrack {{Math}.\mspace{14mu} 13} \rbrack\end{matrix}$

Alternatively, when the third prediction unit 30 performs correction inaffine prediction or in proportion, the r and g in (Math. 1) and (Math.5) can be similarly processed by replacing with the coded pixel signalpositioned in the vicinity of the target block and the representativevalue calculated for every small region from the coded pixel signalpositioned in the vicinity of the block to be coded.

The region of the coded neighboring pixel in the third prediction unit30 may be a predetermined region combining a predetermined number ofcolumns on the left side of the target block, a predetermined number ofrows on the upper side, and the area corresponding to the intersectingarea of the relevant rows and columns and being the diagonal portionwith respect to the target block as in (a3) and (b3) of FIG. 5, or maybe only one portion of such region. It may be a block that contacts thetarget block on the left side or the upper side, or one part thereof. Ifthere is an area that cannot be referenced at the position of the frameend, the referencing area is appropriately switched, or only the portionthat can be referenced may be used. The technique disclosed in NonPatent Literature 2, may be used for the setting of the region.

-   Non Patent Literature 2 Y-H, Kim, B, Choi and J. Pailc    “High-fidelity RGB Video Coding using Adaptive Inter-plane Weighted    Prediction”, IEEE Transactions on Circuits and Systems for Video    Technology, 19, 7, pp. 1051-1056 (2009).

The third compensation unit 31 reconstructs the third prediction signal,and obtains the third prediction signal from the third predictioninformation from the third prediction unit 30 and the coded pixel signal(reconstructed pixel signal) saved in the third accumulation unit 32.The generated third prediction signal is output to the third additionunit 34 and the third differential unit 33.

In the above described example, the prediction signal is generated with(Math. 10) if the third prediction unit 30 uses the quadratic expressionfor the prediction function, and the third prediction information isconfigured from the prediction coefficient a_(j).

The third prediction signal is sent to the third differential unit 33and the third addition unit 34 in the encoder, and only to the thirdaddition unit 74 in the decoder.

Each function block of the image coding device shown in FIG. 1 has beendescribed above. The flow of processes of the prediction referencesignal and the predicted signal in the present invention as describedusing the G signal and the R signal in the examples of FIG. 3 and FIG. 5will be described below. The prediction reference signal of each channelconfiguring the signal is first coded with respect to a target unitblock. When coding the prediction reference signal, the signal in adifferent channel is not yet coded in the unit target block, and hencethe second prediction unit 20 and the third prediction unit 30 cannot beapplied with respect to the input image signal. Therefore, coding iscarried out only in the function block group 100, and the processes inthe function block groups 200 and 300 are omitted.

In the skipping process for omission, processes similar to the processesof when the second prediction unit 20 and the second compensation unit21 are omitted in case when the input first residual signal is flat canbe applied. In other words, the first residual signal is coded in theunchanged value (as second residual signal in form), and at the sametime, the first prediction information and the second predictioninformation serving as information notifying that the second residualsignal has the value of the first residual signal are coded.Alternatively, the information notifying that it is the predictionreference signal and that the first residual signal is used may be addedto the first or second prediction information. The correspondingprocesses thus can be carried out on the decoder side as well.

The reconstructed pixel signal of the target block is saved in the firstaccumulation unit 12 and the third accumulation unit 32, and thereconstructed residual signal of the target block is saved in the secondaccumulation unit 22 by the coding of the prediction reference signal.

The coding of the channel of the predicted signal of the target block iscarried out after such state is obtained. The coding process withrespect to the predicted signal is as described with regards to FIG. 1,and in particular, the reconstructed pixel signal of the predictedsignal itself is saved and used in the first accumulation unit 12(prediction reference signal can also be used). In the second and thirdaccumulation units 22 and 32, the reconstructed residual signal and thereconstructed pixel signal in the prediction reference signal alreadycoded in the target block are used. On each accumulation unit 12, 22,32, the pixel blocks necessary for the prediction is to be held at eachtime point.

According to the image coding device having the configuration describedabove, when carrying out transformation on the pixels of the region tobe coded of each unit block in the transformation unit 1, the spatiallycorresponding differential value of each pixel is transformed based onthe second or third residual signal input from the second differentialunit 23 or the third differential unit 33 by the quantization unit 2,and coded by the coding unit 5.

The residual signal input to the transformation unit 1 is appropriatelyselected from the residual signal obtained through the first predictionand compensation, the residual signal obtained through the firstprediction and compensation and second prediction and compensation, orthe residual signal obtained through the third prediction andcompensation. Hence, high coding efficiency can be obtained by adoptingsignal prediction of predicting the signal of the target block from thecoded pixel signal and reducing the amount of information by adaptivelyselecting different prediction and compensation units.

In correspondence with the description of FIG. 1 made above, eachfunction block of the image decoding device of the present inventionshown in FIG. 2 will be described. The decoding unit 6 decodes the codeinformation coded by the coding unit 5 of the image coding device toobtain the quantization value, and the first to third predictioninformation. The inverse quantization unit 7 and the inversetransformation unit 8 process the quantization value to obtain thesecond or third residual. The inverse quantization unit 7 and theinverse transformation unit 8 have functions similar to the inversequantization unit 3 and the inverse transformation unit 4 of the codingdevice.

The first compensation unit 51, the first accumulation unit 52, and thefirst addition unit 54 have functions same as the first compensationunit 11, the first accumulation unit 12, and the first addition unit 14in the coding device. The second compensation unit 61, the secondaccumulation unit 62, and the second addition unit 64 have functionssame as the second compensation unit 21, the second accumulation unit22, and the second addition unit 24 in the coding device. The thirdcompensation unit 71, the third accumulation unit 72, and the thirdaddition unit 74 have functions same as the third compensation unit 31,the third accumulation unit 32, and the third addition unit 34 in thecoding device. As described in the description of FIG. 1, the output ofeach compensation unit is sent to the addition unit and the differentialunit in the coding device, but is sent to only the addition unit in thedecoding device.

The first, second, and third switching units 80, 81, 82 also havefunctions similar to the first, second, and third switching units 40,41, 42 in the coding device, but the first switching unit has a role ofoutputting the pixel signal sent from the corresponding compensationunit and the addition unit instead of carrying out the switching of thesecond residual signal and the third residual signal as in the codingdevice. As described above, only the third prediction unit 70 in whichthe prediction coefficient is not coded is necessary for the predictionunit in the decoding device, and the first and second prediction unitsare not necessary.

The invention claimed is:
 1. An image coding device for coding an inputimage for every unit block in coding using prediction, the image codingdevice comprising: a first prediction unit for determining firstprediction information for predicting a block to be coded using areconstructed pixel signal reconstructed using a coded signal; a firstcompensation unit for obtaining a first prediction signal of said blockto be coded from said first prediction information and saidreconstructed pixel signal; a first differential unit for obtaining adifference between a signal of said block to be coded and said firstprediction signal as a first residual signal; a second prediction unitfor determining second prediction information as a predictioncoefficient for predicting said first residual signal using areconstructed residual signal reconstructed using the coded signal; asecond compensation unit for obtaining a second prediction signal ofsaid first residual signal from said second prediction information andsaid reconstructed residual signal; a second differential unit forobtaining a difference between said first residual signal and saidsecond prediction signal as a second residual signal; a third predictionunit for determining third prediction information as a predictioncoefficient for predicting said block to be coded using saidreconstructed pixel signal; a third compensation unit for obtaining athird prediction signal of said block to be coded from said thirdprediction information and said reconstructed pixel signal; a thirddifferential unit for obtaining a difference between a signal of saidblock to be coded and said third prediction signal as a third residualsignal; and a switching unit for switching and selecting one of eithersaid second residual signal or said third residual signal for every unitblock, wherein orthogonal transformation, quantization, and coding areperformed on the selected residual, and said reconstructed residualsignal and said reconstructed pixel signal are obtained using predictioninformation corresponding to the selected residual to code theprediction information.
 2. The image coding device according to claim 1,wherein said switching unit compares a coding cost of said firstprediction information, said second prediction information, and saidsecond residual, and a coding cost of said third prediction informationand said third residual to select lower coding cost when selecting oneof either said second residual signal or said third residual signal. 3.The image coding device according to claim 2, wherein said switchingunit carries out the comparison including the cases when the value ofeither said second residual or said third residual is set to zero whencomparing the coding costs, and carries out coding with the value of thecorresponding residual signal as zero if the coding cost of when thevalue is set to zero is the lowest.
 4. The image coding device accordingto claim 1, wherein the reconstructed residual signal used in saidsecond prediction unit and said second compensation unit, and thereconstructed pixel signal used in said third prediction unit and saidthird compensation unit are signals reconstructed based on a secondchannel signal already coded in the block to be coded, separately from afirst channel signal to be coded in said block to be coded.
 5. The imagecoding device according to claim 1, wherein said first prediction unitdetermines said first prediction information as information of an intraprediction mode using an intra prediction.
 6. The image coding deviceaccording to claim 1, wherein said second prediction unit and said thirdprediction unit determine said second prediction information and saidthird prediction information of the block to be coded after obtaining arepresentative value for every small region in the unit block to becoded with respect to said reconstructed residual signal and saidreconstructed pixel signal.
 7. The image coding device according toclaim 6, wherein said representative value is a representative valuebased on filter processing with respect to said reconstructed residualsignal and said reconstructed pixel signal for every said small region.8. The image coding device according to claim 1, wherein said secondprediction unit and said third prediction unit determine said secondprediction information and said third prediction information to minimizea weighted square sum at each pixel position of said second residualsignal and said third residual signal, and set a weighting coefficientof the square sum relatively high for a pixel position referenced in theprediction of the unit block to be subsequently coded.
 9. The imagecoding device according to claim 1, wherein said second prediction unitand said second compensation unit are omitted when said first residualsignal is flat, so that a signal having the same value as said firstresidual signal is adopted for said second residual signal, andinformation notifying the application of the omission is adopted as saidsecond prediction information.
 10. The image coding device according toclaim 1, wherein said second prediction unit and said third predictionunit obtain said second prediction information and said third predictioninformation as coefficients in a polynomial prediction.
 11. The imagecoding device according to claim 1, wherein said third prediction unitobtains said third prediction coefficient using a coded neighboringregion being in the vicinity of said block to be coded and being alreadycoded, only information notifying that said third prediction unit isapplied being coded and said third prediction coefficient not beingcoded when coding said third prediction information.
 12. An imagedecoding device for decoding code information coded by the image codingdevice according to claim 1, the image decoding device comprising: adecoding unit for decoding said code information to said first to saidthird prediction information, and said second or third residual; a firstdecoding side compensation unit for reconstructing said first predictionsignal using a decoded pixel signal and said first predictioninformation; a second decoding side compensation unit for reconstructingsaid second prediction signal using a decoded residual signal and saidsecond prediction information; a third decoding side prediction unit forreconstructing said third prediction coefficient using said decodedpixel signal and said third prediction information; and a third decodingside compensation unit for reconstructing said third prediction signalusing said third prediction coefficient and said decoded pixel, whereinwhen said first and second prediction information and said secondresidual are decoded in said decoding unit, said second predictionsignal and said second residual are added to reconstruct said firstresidual, and the first residual and said first prediction signal areadded to decode the pixel signal, and when said third predictioninformation and said residual are decoded in said decoding unit, saidthird residual and said third prediction signal are added to decode thepixel signal.